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ABSTRACT: This paper presents a model which organizations can use to 
monitor Web site effectiveness. It is argued that anecdotal evidence can be 
colorful but is not useful in structuring and managing an effective site. 
It is suggested that traditional disciplines of composition and 
communication -- explicit purpose, coherent structure, relevant conclusion 
should be applied to Web site design. It is concluded that customer 
feedback must be managed in a disciplined way, by ensuring that feedback is 
representative of the customer population as a whole, not just of those 
with a propensity to comment; and that the purpose and aims of a Web site 
must be thought through with the utmost care and attention to give a higher 
likelihood of creating an effective site. 

TEXT: Abby Day: Calgary, Alberta, Canada 

How do you know if your Web site is doing its job? Hits per day? Sales per 
week? All or none of those may matter, but how will you know which ones, 
and how will you find out? The purpose of this paper is to present a model 
which organizations can use to monitor their Web sites. First, a few common 
misconceptions are exposed. 

Literature on Web theory is scant, for two reasons. First, it is a new area 
and many people are simply finding their way for the first time. Second, 
those who are at the forefront of the Web, the design technologists, are 
not typically inclined to sit back, reflect on their practice, source 
relevant theory and write about it. 

As a result, what exists is either anecdotal or prescriptive. On one hand, 
there is the "I use loads of graphics and it works well" school; on the 
other hand there is the "don't use more than three icons per page" school - 
each equally unhelpful. 

Anecdotal experience may be fun and colorful, but it does not build a 
picture or a pattern for others to apply to their own experience. Offering 
prescriptive statements such as "too many/too few icons are bad" is a 
little like saying "blue cars are bad." They are not bad or good, they 
either work or they do not. But why they work is not a function of the 
icons, the graphics, the content, the color, the length of the page, the 
number of links, the speed of my modem or any other technological variable. 
A Web site works because the people it serves like it. 

No one, yet, is forced to use the Internet. We may argue that it helps in 
our research or our businesses, but we do not, yet, have to use it. We will 
therefore return to sites for the same reason that people buy from certain 
salespeople - they like them. 

This article explores likeability factors and suggests ways that those 
factors can be controlled and their effectiveness monitored. It is also 
premised on the same variable that we use to determine whether an 
organization is succeeding in its objectives - the purpose is not to have a 
one-time-only interaction with our customers, but to build a long-term 
relationship where customers return again and again. 

Web ennui 

The first step in building a model of Web effectiveness is to debunk the 
myth that the Web is interesting. The only people who find the Web 



interesting are the people who program it, but then, they think computers 
are interesting, too. The rest of us find the whole subject of bits per 
second and PDFs and GIFs a real bore. What we really want to know is how to 
get the weather in Chicago or if anyone else in the world is doing research 
on long-range planning in Turkey. 

That is not to say that serendipity is not part of the charm. Idle, 
non-linear browsing may form part of our Web activity, but it is not 
something a single site-owner can control. Were there a body such as 
Internet Inc., then the model could be applied to the Internet as a whole, 
but there is not. That is why the shopping mall analogy so commonly used is 
weak. Someone does manage a shopping mall, and assures that the individual 
stores comply with the integral design. We cannot do that for the Web as a 
whole, but we can for our own sites. 

Our model for Web effectiveness does not incorporate any features of the 
process. How it works is up to the technologists: we just want to see if it 
works and why. It is a little like taking a flight from Toronto to London. 
We do not really care how the airplane stays up in the air, nor do we want 
to know it is built on something that is still called a theory - we just 
want to arrive refreshed and cheerful, without falling into the pond. 

Message, not medium 

The second step is to debunk the myth that the Web represents a completely 
new form of communication which needs new rules. This myth was probably 
generated by computer technologists whose strengths do not usually lie in 
verbal communication - with a few notable exceptions. 

Everything we need to know about Web communication we learned in grade 
five. Every effective communication has three distinct parts which we will 
explore in more detail further on: 

- an explicit purpose; 

- a coherent structure; and 

- a relevant conclusion. 

That we use words, graphics or a mix of the two does not matter. Even comic 
books have titles, logical structures, and resounding finishes. Some pretty 
awful Web sites look like the kind of brochure created by an office clerk 
using the latest desktop publishing software. Just because we can push 
buttons to create different images does not mean the final product will 
hang together with integrity. Graphic designers know how to arrange text 
and visuals to complement each other , and to influence the audience. They 
have rules about weightings and fonts and sizes - all of which combine to 
create a professional product. 

If we do not follow the few, simple rules about effective communication 
then we will fail in our message, whether the medium is a book, a Web site 
or a television show. To monitor our site's effectiveness, we first have to 
ensure that we have the basics of good communication working for us and our 
audience. It is the one variable which will undoubtedly cause our failure 
if we get it wrong, no matter how fast, sleek and sexy our site might be. 

All of this is about doing the right things well, rather than doing the 
wrong things more quickly to more people more often. It is the difference 
between effectiveness and efficiency. As someone once said about using a 
spreadsheet for forecasting: "if I'm not careful I just compound my errors 
faster. " 

Quality is ... 

Quality is whatever the customer says it is. Beyond the basics of keeping 
the wings on the plane, engineers do little to influence the 
moment -by-moment quality experience on an airplane. That is down to the 
standard of service performed by the cabin crew, the taste of the food, the 



variety of entertainment on offer. 

Keeping the wings on the plane is static quality: we do not want it to 
change and we only really notice it if it stops working. Offering hot 
towels or offering to walk your crying baby is dynamic quality; we notice 
it because it attends to our individual needs in a unique and completely 
relevant way. It may change from airline to airline, whereas keeping the 
wings on probably will not, we hope. 

A model for Web site effectiveness must therefore presume that we 
understand who the customer is and what the customer will see as quality. 
The model does not presume an internal committee of designers who judge the 
site - that would be a little like those notorious *advertising* awards 
where all the agencies congratulate each other on creative design, but no 
one says whether the ads worked or not . 

Quality is customer-f ocussed; it therefore must be customer-led. Putting 
the customer at the center assures that our communications mirror the 
customer's needs and preferred style of medium. "We need to establish the 
basics of static quality and then be responsive enough to offer dynamic 
quality as well. We need to do some of that alone, and some of that with 
our customers . 
Owning up 

What then, are " the elements of static quality which we can firmly 
establish, which will not change daily and for which we must take utmost 
responsibility as professionals? We need to identify those and then monitor 
that they work in practice. These are the elements that do not depend on 
customer interactivity. They are what is known in strategy theory as 
"conditions of doing business". They are not options and they are not open 
to daily review. These elements are to a Web site what knowing protocol is 
to a diplomat - they will not guarantee you will win the war, but without 
them you will not even get an audience. 

The second element, of dynamic quality, is the only element which we can 
offer for customer feedback. This may change on a weekly basis, but we can 
manage the change within a model . 

Let us explore the elements of static Web quality which are our 
responsibility and see how we can monitor them: 

(1) Identifying our customers. 

(2) Articulating the site's purpose. 

(3) Providing relevant content. 

(4) Creating coherent structures to mirror customers' needs. 

(5) Arranging visual elements to reflect (1), (2) and (3) above (we call 
it housestyle) . 

(6) Creating relevant concluding/action points. 

(7) Delivering the promises we made at the action points. 
Customers 

If we know why we are in business, we must know who our customers are. Just 
because the Web reaches millions does not mean they are all our customers, 
any more than saturating a city with leaflets means we will get to the 
people who will eventually buy. We may hit them in passing, but a message 
which appeals to everyone will be so diluted it will not appeal to the 
group we want . 

Who are the customers in your business and what are they doing on the Net? 
Why are they there and what do they hope to gain from being there? Those 
are questions which need to be answered before we can design the words and 



pictures to satisfy them. Any medium of mass communication, whether that be 
television or newspaper or Web, can seduce us into thinking we are 
communicating with the people we want. Leaders in the direct mail industry 
learned a long time ago that the most important element of the 
communication was the * target* audience. 

We may not prevent everyone from visiting at least the opening page of our 
site, but we are not going to put any effort into satisfying everyone's 
needs. We can monitor this by evaluating the nature of the people who go 
further into our site, even as we create reasonable obstacles to impede the 
progress of the casual observer. The last thing we want to do is to clutter 
our database with thousands of people who have no real interest in what we 
do. 

That obstacle may be a registration page, it may be the need to pay money 
or it simply might be the specificity of the content. Either way, we want 
to measure the quality of the visitors and see if it matches the profile of 
our customers . 

Purpose 

Every effective communication has a clearly stated purpose known to both 
the creator and the audience. Good articles begin with the purpose (Day, 
1996) ; good presentations begin with a purpose; good marketing 
communications begin with a purpose. A Web site is no different. We must 
tell our customers immediately why the Web site is there and what it will 
do for them. 

The problem is often defining the purpose in the first place. Many Web 
designers fall into the same trap as other poor marketing communicators - 
they feel that their presence alone is purpose enough, that "awareness" is 
sufficient. With all the investment required to create and maintain an 
effective site, simply being there is not going to provide a return. That 
return does not have to be measured in pure financial terms - it may be 
sufficient to say that our purpose is to attract other researchers, or to 
create a database of prospects. But it must, like any marketing 
communication, be quantifiable. 

Too many people measure their site effectiveness by the number of "hits" - 
that is, by the number of people who log on. That is like measuring the 
effectiveness of a television commercial by the size of the audience. 
Granted, some people still do that, but not the effective ones. What we 
want to know is: are the right people visiting our site and are they doing 
what we hope they will do when they are there? 

That needs hard thinking, but no one said it would be easy. As a first 
step, ensure the purpose is distinctly clear and has a quantifiable 
customer- focused measure relevant to the objective. Then, monitor it using 
that criterion only. 

Content 

The question of "what should be on the page" is answered by understanding 
the customers' needs, behaviors and expectations, combined with the purpose 
of the page. 

Some organizations may be offering text -based papers or conferences; others 
will be enticing customers to buy lawn furniture . Others are just an 
expression of the Web-owners' ego ("Hi, I'm Dave, welcome to my homepage") 
which do not concern us here. 

What we have discovered so far from the Internet is that people behave 
pretty much as they do in other environments. The extrovert takes the lead 
in a discussion group; the introverts lurk. The person who wants something 
for nothing comes and goes; the community-minded person contributes and 
enhances the site by their presence. 



Offering content for customers will be based on your understanding of who 



they are and how they will behave. But how do you know if you have it 
right? This is one area where involving your customers is critical. Test 
the content on a * target* group - we will discuss how to do this later on. 

Structure 

Structure follows strategy. Once the above three items are worked out - the 
customer group, the purpose, and the content - then the structure is easy. 
Most confusing Web sites do not follow a logical structure because they are 
not created with the logic of the customer in mind, or follow the logic 
defined by the purpose. Creating a coherent structure is a function of two 
important variables: 

- following the logic dictated by the customer and purpose; and 

stating clearly along the way what you are doing, commonly known as 
"signposting" . 

When I run authors' workshops at universities, I distribute at random 
journals of different disciplines, then ask the participants to choose one 
paper at random. They then have five minutes to discover what the purpose 
is, the key points and the implications. If they cannot do that, the 
article is deemed a failure. Anyone should be able to skim read a paper 
about anything and at least be able to state the purpose, main points and 
implications. At least, anyone looking at a paper which has been written 
properly. 

That does not mean we have understood the paper in depth - that may take 
several readings. But just as we can scan titles and abstracts to decide 
whether we will read the paper, so we can skim the paper and decide whether 
it is worth reading again, slowly. 

The same situation occurs with a Web site. Anyone should be able to find 
the site, see instantly what it is about and what it has to offer, navigate 
quickly to key pages, understand what they can do next and decide whether 
or not to do it. That the site offers a multiplicity of options is not the 
point. Those options and their relevance should be instantly clear. 

Now, how do we know whether or not the structure is working? If we know the 
customer group and the purpose of the site, then we must assume we can 
predict the customers' behavior to some extent. We must have a theory about 
what the customer will do when faced with a certain page at a certain time. 
I am assuming, of course, that we have already dispensed with those people 
who are not our customers by weeding them out early on in the process. But 
for those we know and understand, we should have a theory about what they 
are going to do. 

It is exactly the same with placing an ad in a magazine showing a certain 
picture, having a certain headline, and presenting a certain offer. Our 
theory is that a certain sort of person seeking certain benefits will 
respond in a certain way to the offer and respond accordingly. That is how 
all effective marketing communications work, grounded in such a theory, and 
our Web site is no different. 

We can apply the *advertising* industry's short guide to effective 
communication design: 

- attention; 

- interest; 

- desire; and 

- action. 

We can therefore monitor the structure's effectiveness in two main ways. 
First, we can observe the behavior of those entering the site and starting 
to move through it. We can track their movements and see if they go to the 



areas we want them to go to. We can see if they consistently log off at a 
certain point and then tie that behavior back to our theory. If we had 
assumed that our customers would want to download that hefty file of 
information because we know it will match their interests and yet we see 
that most customers do not, we can safely assume that the average download 
time of three minutes may be too long. So, we can cut that file into 
smaller pieces and see if they hang on for 15 seconds. 

We can help them along, just as we do in an article, by advising them of 
what is to come and how long it will take. 

The second prime way we can track their behavior is to see whether they 
return to the site. The fundamental objective of any organization is to 
have repeat business - few people survive by interacting with customers 
only once. How many of our customers come back again and again, and how do 
they move through the site? We all know about bookmarks, and we usually 
bookmark the sites we want to return to. By measuring the return hits and 
looking at the behaviors we can quickly turn those data into useful 
information: 20 percent of first-time users come back again; 15 percent of 
them consistently log off at certain points, for example. That gives us 
the best information of all - it tells us what the people we are really 
interested in are doing. 

Housestyle 

The Web site must have a housestyle, just like a publishing house has for 
its books and journals. No one claims that the choice of housestyle 
elements makes the biggest difference for the audience (it actually makes 
the least), but it must be appropriate to the audience's needs. 

Having an aesthetically pleasing site is not an art form anymore; it is 
just another condition of doing business. But it must be grounded in the 
theory. Decisions to use a certain typeface or certain graphics are driven 
by our understanding of the customer and the purpose of our page. 

There are, however, simple design conventions which need to be followed on 
a Web site just as they are on a printed page. The few rules are once again 
the kind of static quality which needs to be embedded in the site. The 
usual rules governing housestyle are: 

- (1) Integrity. The material, whether it is a Web site or a book, needs to 
have consistency. Once our audience has warmed to a certain style they 
begin to depend on it. Unfortunately, many Web sites appear to suffer from 
innovation overload, where a variety of designers have free rein to design 
whatever pages they like within the same site. That would be like having 
different typefaces for different chapters of a book. 

This may be what Artz (1996) describes as the result of an "evolutionary" 
approach to Web development, as opposed to a top-down methodology. He 
argues that the weakness of the evolutionary approach is that participants 
are all involved in a joint experiment or learning experience: 
"consequently, evolving prototypes often suffer from several common 
problems including failure to meet business objectives; design conflicts 
resulting from conflicting purposes; inconsistent look and feel; data 
synchronization problems; data update problems; maintenance difficulties; 
and complex and confusing interfaces." 

If these problems sound familiar, perhaps it is time the organization 
accepted a top-down methodology. Applied to housestyle, that would mean all 
designers would know, for example, that each page must start with an 
introduction; it will contain horizontal rules every three paragraphs, two 
image links, a "last-updated" note and a feedback e-mail section (or 
whatever is deemed appropriate to the customer group and to fulfill the 
purpose) . 

- (2) Clarity. A good lay-out assists comprehension. Every graphic designer 
knows that minimizing variety helps the reader, just as crowded text and 
pages turn them off. Conventions governing how text is laid out and when 



' images are used becomes part of the housestyle to be followed by each 
designer. An organization without a complete understanding of what governs 
clarity needs to get back to the drawing board before it launches itself on 
the Web. 

Once again, I am not proposing that a new approach to this medium is 
required. What I do suspect, however, is that the same measurement will 
apply here as it does to ads and books: the creative execution is there to 
assist the reader, not exist for its own purposes. As long as it is good 
enough, it's good enough. But if it is poor, then audiences will be lost 
forever. 

The housestyle therefore determines that good practice is followed, 
creative designers are managed to the brief rather than to their own whims, 
and everyone knows what practice to follow. That allows us to move to the 
meat of the site, the content, without interruption. 

Creating and maintaining housestyle also means we can remove one variable 
when monitoring behavior. We will not be confused by a page which is 
prettier than others, or a page with no graphics, when we are trying to 
assess the site's likeability. 

Action 

As every communication has a purpose, so it must have a result. What do we 
expect will happen as a result of our communication? This goes back to the 
points of purpose raised earlier on, and impacts on the design of the site 
itself. And yet, something so simple as action points may be overlooked or, 
worse, degraded to the "give us your feedback" pleas. 

We are not on the Web to gain feedback. We are on the Web because we 
believe something will happen as a result of our presence. Just as we would 
measure our organization's success based on number of products sold, or 
number of students registering, or number of votes at election time, so we 
must set a measure for the success factors we will attribute to the site. 

How easy are we making it for our *target* group to do what we hope it will 
do - to place an order, to fill out a form, to add to a conference, to 
request further information? Again, this is an enormous difference from 
simply measuring "hits". Just as our purpose cannot be as sloppily defined 
as "awareness," so our action measures cannot be a function of the number 
of people dropping in. 

The measure needs to be quantifiable and built into the strategy of the 
site. It then needs to be monitored and reviewed, just as any 
organizational activity should be, and decisions made as a result. 

Promises delivered 

The only thing worse than not promising results is making promises and not 
delivering them. A critical part of the Web site's effectiveness is how it 
performs its objectives, and some of those will be linked to other parts of 
the organization. 

For example, the Web site may be successful in attracting its ^target* 
audience and convincing the audience that it is worth the effort to fill in 
the form and request further information about the doctoral program, the 
running shoes, the cruise, or whatever. 

What happens next? How is that information fed through to different parts 
of the organization, and who determines that it is acted on? Many 
organizations see their Web site activity fall into a black hole at this 
point, or come up against a brick wall. The very parts of the organization 
which need to act on the customer's request may be those least inclined to 
embrace electronic marketing. Who takes responsibility for action to 
happen? 

Before making empty promises, the Web site champions must walk themselves 



mentally through the processes which will occur once the customer presses 
an action button. Map the flows from this point and see where they lead. As 
they lead away from the IT branch, where do they go and on whose desk will 
they land? Will the request for further information go into the same piles 
as all the other requests? 

The customer on the Web is accustomed to something the print -based customer 
is not: speed of response. We know by their * inclination* to use the Web 
that they want and expect instant responses - look how they complain when 
their system crashes or someone's site takes too long to load. We are 
dealing with a * target* audience with a certain level of expectation, which 
must be met at each stage. 

Once you have walked through the action flows, there might be another job 
to do before the potential problems can be resolved. These are serious 
matters of organization structure and politics: someone, most probably at 
the most senior level, needs to approve the cause and help remove barriers 
along the way. 

Creating customer focus groups 

Asking for feedback is largely a hit and miss affair. Customers expect to 
have their needs fulfilled by exploring your Web site; they are not in the 
business of charitably offering their serves as a market research group and 
it is unrealistic to expect them to do so. 

Even so, some do respond. But, does that mean they are representative of 
the customer group you are seeking to *target*? Those who offer feedback 
may be a category of themselves: " f eedbackers , " just as those who use 
coupons in stores are a category of themselves: "coupon-users." 

To create meaningful feedback we need to concoct a group representative of 
the whole. This is essentially what anyone does when they conduct market 
research. The sample must be a sample which can safely act as a surrogate 
for the group at large. We are, of course, assuming that we know who the 
*target* group is, know them intimately enough to select members who will 
give us information we can apply to the rest of the group. 

The sample group must be invited to act as a market research sample, and 
induced to do this through an appropriate reward system. Depending on the 
group, these rewards may take the form of prestige factors, cash, product 
samples, or gifts - all of which need to have *affinity* with the product 
or service we are actually offering. 

We will ask our group to comment on the key variables of the site we need 
our most important feedback on: the design and the content. The questions 
we ask need to be appropriate to the benefits we assume our group is 
seeking. For example, if we believe interactivity is the main benefit, then 
we need to ask our sample group to comment on that factor. If we believe it 
is the nature of our up-to-the-minute industry information, then we will 
ask about that. 

The point is, there is no sense in asking our group open-ended questions 
like "what do you think of our site?" Our questions need to be directed 
toward the criteria we think are most important, and only those. 

Implications 

This article has provided a wide view of the Web site's role and potential, 
stressing that measurements must be struck which are relative to the 
organization's and *target* group's objectives. These can then be monitored 
against those criteria. 

Further, the paper has proposed that a Web site must be championed at the 
highest organizational level to ensure that the action the customer wants 
is acted on. 



The implications for further research lie in the areas of profiling actual 



customer behavior and seeking correlation between predicted and actual 
behavior; reducing the number of variables through consistency to allow 
effective measurement; and the impact of the medium on organizational 
structure and culture. 
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